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(54) Load test system and method 

(57) A system and method for load testing software 
applications is provided. The user interface and/or 
application calls (256) are captured by a capture agent 
(262) to generate a script (264) to emulate a user ses- 
sion. The saipt may include source language state- 

252 
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ments and. with or without editing, may be compiled into 
an executable script Multiple scripts m^ be executed 
on a 8crq3t driver to simulate multiple users to load test 
a system. 
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Description 
COPYRIGHT NOnCE 

A portion of the disclosure of this patent document 
contains niaterial which is subject to copyright protec- 
tion. The copyright cwner has no okjedion to the xero- 
graphic r^roduction by anyone of the patent document 
or the patent disclosure in exactly the form it appears In 
the Patent and Trademark Office patent file a records, 
but otherwise reserves all copyright rights whatsoever 

BACKGROUND OF THE INVENTION 

The present invention relates to the field of software 
systems. More specifically, in one embodiment the 
invention provides an improved method of load testing 
software and. especialty, a system and method for 
scripting user actions for a load test 

There are a number of different approaches to load 
testing. For example, live users may be utiized. The 
system developer hires live users and buys the hard- 
ware required (PCs or termtnats) to drive the system. 
This approach, however. Is very expensive. In otter to 
cut costs, someone considering this approach usually 
decides to test using some fraction of the actual number 
of users that the actual system will be required to sup- 
port afxl to interpotate actual response time figires 
based on the response time of the fraction. This method 
tails largely due to inadequate testing of threshold con- 
ditions. Also, even If the entire number of proposed 
users can be seated and instmcted to perfonn the test 
actions, it will be impossible to control their rates of input 
or accurately measure the response times. 

Simulations have also been used The developer 
computes the theoretical load imposed upon the system 
and interpolates the response times and ttieoretical 
number of users that the system may support T^s 
method has very litUe basis in reality and provides no 
confidence that its assumptions are correct or Hs results 
accurate. 

Canned benchmar1<s may also be utilized. A 
number of pubGdy available benchmark tests are avail- 
able that exercise a piece of hardware and the operating 
system that runs it. but this does not alicw for ttie testing 
of a particular application. 

The problem addressed by the invention is to pro- 
vide an improved system and method for load testing 
software applications which may overcome tiie limita- 
tions In the conventional systems and methods 
described afc»ove. 

SUMMARY OF THE INVENTION 

A system and method for load testing software 
applications are provided by virtue of the present inven- 
tion. The system interjects a Captire Agent that may 
capture or intercept user interface and application calls 
in order to generate a higher-level script The saipt 



atong with other scripts, may be compiled and executed 
to simulate multiple users to bad test software applica- 
tions. In first aspect of the invention, a method of pro- 
ducing scripts for load testing a software application in a 

5 dient/server environHoent includes the steps of: captur- 
ing application calls on the client computer, the applica- 
tion calls including application calls generated in 
response to the captured us^ interface calls ; recording 
timing Information of the captured application calls : and 

w generating a script from tiie captured application calls 
that generates application calls according to the timing 
information of the captured application calls. Addition- 
aDy, user interface calls and associated timing informa- 
tion may be captured and incorporated into the script In ^ 

15 a second aspect of the Invention j a metiiod of produc ing 
scr^ for load testing a software appTication in a cG- 
ent/server environment, indudes the steps of: capturing 
application calls on the client conrputer. the captured 
application caNs Including references to data stored 

20 locally on the dient computer: identifying in the cap- 
tured application calls references to data stored locally 
on the client computer: accessing ttie data through th e 
references in ttie captured application calls: and gener- 
ating a saipt from the captured application calls that 

25 generates application calls that indude ttie accessed 
data In place of ttie references in the captured applica- 
tion calls, the script including the accessed data. 

In a third aspect of ttie invention, a mettKxJ of pro- 
ducing scripts for toad testing a software application in a 

30 dient/server environment includes ttie st^ of: captur- 
ing application calls on ttie client computer, ttie captured 
application calls specifying information to be sent to the 
server computer: translating the captured application 
calls into source language statements: and generating a 

35 script induding ttie source language statements that 
generates appfication calls. Additionally, ttie script may 
be user edited or compiled to produce an executable 
load test program. 

In a fourth aspect of the invention, a m^od of for 

40 load testing a software application in a networked com- 
puter system having a first computer (scr|>t drh/er) and 
a second computer (system under test), includes the 
steps of: the first computer executing sajpts that emu- 
late a pluraiity of users, the scripts including delays rep- 

45 resentative of actual user sessions: the first computer 
sending requests to ttie second computer over a net- 
woric the second computer responding to the requests 
over ttie networic and measuring response times of ttie 
second computer. Additionally, a ttiird conputer may be 

50 on ttie networic to display a script cfirected user session 
In progress. 

A forttier understanding of ttie nature and advan- 
tages of ttie inventions herein may be reafized tsy refer- 
ence to ttie remaining portions of ttie specification and 
S5 ttie attached drawings. 

ppigp ngSCRIPTlQW OF THE DRAWINGS 

Fig. 1 illustrates an example of a computer system 
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that may be used to execute eoRware acoording to 
an embodiment of the present invention: 
Rg. 2 shows a system block diagram of a typical 
computer systm: 

Rg. 3 illustrates a client/server architecture: s 
Rg. 4 shows a high level flowchart of a typical data- 
base application: 

Rg. 5 shows the data flow of an embodiment of the 
invention; 

Fig. 6 shows a high level flowchart of load testing; lo 

Rg. 7 shows load testing of a system under test: 

Rg. 8 shows a flowchart of tfie process of aeating 

an executable load testing progmm; 

Rg. 9 shows a high level flowchart of the Capture 

Agent; 

Rg. 10 shows a process the Capture Agent per- 
forms to generate a script from the capture calls: 
Rg. 11 is a typical captured database session: 
Rg. 12 shows processing of a database function; 
Rg. 13 shows a flowchart of perfbrming load test- so 
ing; 

Rg. 14 shows load testing of a system under test 
suitable for display or non-display mode; and 
Rg. 1 S shows a flowchart of a process of emulating 
a user session from a script ss 

DETAILED DESCRIPTION OF PREFERRED EMBOD- 
IMENTS 

Ptfirwtipng: ^ 

System - a combination of softifvare and the hard- 
ware It runs on. 

Load Testing - the process of analyzing the effect of 
many users on a system. ^ 
Response Time - the time between a user-initiated 
request and the response from the system to that 
request 

Client/Server - a computer architecture where a 
networked server computer services client conput* 40 
ers that are intelligem. programmable devices. 

Load Testing is an essential step in the application 
developmeht process. It allows a developer or systems 
integrator to detemiine the perfbrmanoe and scalability 45 
of a system prior to the deployment of the system 
measuring the user-perceived response time. The 
present invention utilizes user emulation to load test 
software applications. This process emulates live users 
by perfornwig the activities of the actual users, prefera- so 
biy in a nnanner such that the actual system cannot dif- 
ferentiate between the emulated users and the actual 
users. 

Rg. 1 illustrates an example of a computer system 
that may he used to execute software of the present 55 
invention. Rg. 1 shows a computer system 1 which 
includes a monitor 3. screen 5. cabinet 7, keyboard 9. 
and mouse 11. Mouse 11 may have one or more but- 
tons such as nmuse buttons 13. Cabinet 7 houses a 



CD-ROM drive 15 or a hard drive (not shown) which 
nrtty be utilized to store and retrieve software programs 
according to emtxxliments of the present invention, 
data for use with the embodiments , and the Oke. 
Although a CD-ROM 17 is shown as the removable 
media, other removable tangible media including floppy 
disks, tape, and flash memory may be utilized. Cabinet 
7 also houses familiar computer components (not 
shown) such as a processor, memory, and the like. 

Rg. 2 shows a system block diagram of a typical 
computer system. As in Rg. 1, computer system 1 
includes monitor 3 and keyboard 9. Computer system 1 
further includes sUbsystenns such as a central proces- 
sor 102. system memory 104.- I/O controller 106. display 
adapter 108. removable disk 112. f'wed disk 116. net- 
work interlace 118. and speaker 120. Other computer 
systems suHatsle for use with emt>odiments of the 
present Invention may include additional or fewer sub- 
systems. For example, another computer system could 
include more than one processor 102 (i.e., a mufti-proc- 
essor systm) Of a cache memory. 

Arrows such as 122 represent the system bvs 
architecture of computer system 1. However, these 
arrows are illustrative of any interconnection scheme 
serving to link the subsystems. f=or example, a local bus 
coukf be utilized to connect the central processor to the 
system memory and display adapter. Corrpute^ system 
1 shown in Rg. 2 is but an example of a computer sys- 
tem suitable for use with embodiments of the present 
invention. Other configurations of subsystems suitable 
for use with embodiments of the present invention will 
be readily apparent to one of ordinary skill in the art. 

Rg. 3 illustrates a dient/sen/er architecture. A cli- 
ent/server system 130 includes a first computer or 
server 131 and one or more second computers or cli- 
ents 150. Typically, the clients 150 are connected to 
server 131 through a computer network 1 41 . which may 
be a conventional Ijocal Area Network (LAN). Network 
141 includes C£d>ling 145 for connecting the server and 
each client to the network. The clients themselves may 
be similar to or the same as computer system 1. Each 
dient typically Indudes a network connector or adapter 
143 for receiving the network cable 145. as is known in 
the art. Server 131 may also be similar to or the same 
as computer system 1. Because the server manages 
multiple resources lor the cfients. it shouki preferably 
indude a relatively faster processor, larger mass stor- 
age, and more system memory than is found on each 
dient. 

Overall operation of the system 1 30 is directed by a 
networking operating system 137. which may be stored 
in the server's system memory. In response to requests 
from the dients 150. the server 131 provides various 
network resources and services. For instance, multiple 
users (e.g.. dients A. B and C) may view a database 
table stored in fSe server storage 139. while another 
user (e.g.. dient E) adds a database record. 

The following description will focus on a preferred 
embodiment of the present invention, where the dient 
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computers are IBM compata^le computers running Win- 
dews 3.1 and the script driver is a UNIX workstation 
(e.g.. from Sun Miaosystems. Inc.). The server pro- 
vides database functions for a database application lite 
those available from Orade Corporation or Sybase. Inc. $ 
The network operating system that provides network 
communication may be from a vendor such as Novell. 

The present invention, however, is not limited to any 
particular applicalion or any particular environment. 
Instead, those skilled in the art will find that the teach- io 
Ings of the present inventk>n may be advantageously 
applied to a variety of other applications including 
spreadsheet appfications. word processors, and the 
like. Moreover, the present inventton may be embodied 
on a variety of different platfonms, irv^uding Macintosh, is 
NextStep. and the like. Therefore, the desatption that 
follows is for purposes of illustration and not limitation. 

One difficulty encountered in user emulatk>n is the 
problem of how to devetop a test script that accurately 
represents a user's activities and rates of input to the 20 
system being tested. This problem is complicated when 
the system to be tested is a part of a dienVserver appli- 
cation, where the client part of the system is located on 
a personal computer (PC) and the server part on 
another, usually larger, host. 25 

Systems according to embodiments of this Inven- 
tion inrpiement a method of capturing user activities that 
are part of a diem/server system. This method works in 
a way that accurately records the userls rates of input 
dient delcQ^s caused by local processing of data, and all 30 
other inputs necessary to con-ectfy replay and replicate 
that user's activities to perform a k>ad test. Although the 
particular implementation is specifically developed to 
capture particular commercially available database 
user*6 activities (Orade. Sybase, etc), it can be modified as 
for another database or fbr another type of cHent/isen^er 
application. It can also t>e modified for another dient 
SKle operating system such as OS/2. Windows BS. or 
Windows NT. 

In a prefenred embodiment, a single nuichine may 40 
be used to accurately slnrulate hundreds of users. The 
system creates scripts that represent actual users and 
their daily, often disparate, operations. With the Capture 
Agent, the system records usa* activities, induding key- 
strokes, mouse movements and SQL requests, to ere- 45 
ate emulation scripts. The system then arranges a mix 
of scripts that represent actual users. The system 
rafeals if a software applk:atk>n or system under test 
works. Before deployment, one can correct common dif- 
ficulties that emerge during the application development so 
process. The system may optionally chart the time one 
waits for screen responses, and ma^ find hidden bugs 
and bottlenecks. 

In a prefen-ed embodiment, the Capture Agent cap- 
tures Windows and SQL Application Progranming ss 
Interface (API) calls. These user interface and applica- 
tion calls may be captured or intercepted by an in-menv 
ory replacement of API caO addresses with Capture 
Agent function addresses, by a renanning of the API 



library on disk so that the Capture Agent is called in 
place of the API library which then calls the renamed 
library, hooks, or other mechanisms. As will be dis- 
cussed in reference to Fig. 5. the Capture Agent moni- 
tors the API calls and generates a script that 
encapsulates the cads into a form that may be executed 
on another host (or script driver) This is unique in that it 
is not merely a trace of the API calls. The calls to the 
API are nrmitored and re-written into a higher-level or 
source language that allows the data and a representa- 
tion of the API calls to be recorded into a procedural 
script that can be compiled and executed. Among other 
things, this requires references to program variafcile 
addresses to be resolved by the Capture Agent and the 
data refen'ed to to be recorded in the script. 

Another unkiue part of this process is that while the 
application calls (e.g.. SQL API calls) are being moni- 
tored, the user interiace calls (e.g.. Windows API calls) 
of the operating system are also being monitored so 
that the user's activities may be recorded. These activi- 
ties or events indude nxuise motk^ns, button presses, 
key presses, as well as user-generated delays. The 
process differentiates between delays caused by the 
user (due to a user pajse). delays caused by the cfient- 
slde of the applkation (due to some internal processing 
of data, for exan^e). and delays caused by the sen^er- 
skJe of the application. Since the goal is to accurately 
emulate all of the activities of a dient these delays are 
important to the emUlatton of the user. Discardir^ 
delays caused t>y the internal processing of data (for 
exarrple) will cause a higher bad to be imposed on the 
server due to faster arrival times of requests. 

Fig. 4 shows a high level f towchart of a typical data- 
base application. The database application may be from 
such vendors as Orade Corporatton or Sytxise. bic. 
Although the dient/server application desaibed is a 
database application, other applk:ations may be utilized 
with the present invention. For example, spreadsheet 
applicatk)ns. word processors, or any other dient/sen^er 
appik»tk)n. 

Once a database applicatton is running, the system 
receives user input at step 202. The user input may be 
any type of user input like typing characters on the key- 
tx>ard or moving the mouse and "diddng" (depressing a 
mouse button) on a menu selectkm in a Graphical User 
Interface (QUQ. The operating system typically receives 
these user events and makes a user interiace call to the 
appropriate application which, in this case. wHl assumed 
to be the clatat>ase application. 

At step 204. the database application receives the 
user interfeice call from the operating system. The data- 
base application then processes the call to determine 
what action, if any, should be taken. The following 
describes just a few of the posstole user interiace calls 
that may be received. The calls are not intended to be 
an exhaustive list and may vary from application to 
applicatk)n. Nevertheless, the calls are intended to aid 
the reader's understanding of a typical database appS- 
catk)n. 
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At Step 206, the datat)ase application determined 
that the user interlace call Is a request to logon to the 
database application. A bgon lyisically includes the 
user's nanne and a passwofd. The datat>ase application 
Jhen verifies and stores the logon informa ffiir 

At step 208. the database application received a 
user interface call requesting to insert data into the 
database. After the database application verifies the 
request the database application makes an application 
call to put the data In the database at step 210. The 
application caO is sent over the network to the database 
server. 

At step 212. the datat>ase application received a 
user interface call requesting to fetch data from the 
datak)ase. The database application makes an appropri- 
ate appficatlon call to get the requested data from the 
database at step 214. The application call is sent over 
the network to the database server. 

At step 216. the database appHcation determined 
that the user interface call is a request to logoff the data- 
base application. After each of the IbUowing user inter- 
face calls the system receives more user input at step 
202 which may be passed on to the database applica- 
tion. However, if the user interface call specifies that the 
user desires to exit the datafc>ase appiicaton. the data- 
base application exits at step 218. 

Fig. 5 shows the data flow of one ent>odiment of 
the invention. A client computer 252 is linked to a server 
computer 254 over a network In the embodiment 
shown, the dient is running Windows as an operating 
system and the database application is an SQL data- 
base like Oracle. A user interacts with the client through 
a user interface 256, here shown as Windows. As the 
user types characters on the keyboaid or operates the 
GUI, these user events generate the appropriate user 
Interface call in the form of a Windous API call to a client 
application 258. The dient application is a front^nd for 
the datatiase application, which is shown as an SQL 
server. The front-end client applicatk>n utilizes the 
processing power of the dient computer in order to 
• offkMKJ sorne of the processing required from the data- 
t>ase server. Thus, the database application has a di- 
ent-side application and a server-skJe application. 
Typically, the client applicatk)n is optimized for user 
interaction and the server application of the database 
application is optimized for servicing multiple users to 
take advantage of the dient/server architecture. 

The dient application receives user interface calls 
and sends the appropriate application calls in the form 
of an SQL API call to a network transport layer 260. The 
network transport layer is typicaly a driver that formats 
the call for transmission cyver the network to the server. 
Similarly, the network transport layer receives data from 
the server sent to the client. 

An important aspect of the present emt>odiment is 
the Capture Agent. A Capture Agent 262 captures one 
or both of the Windows and SQL API calls during a user 
session. The Capture Agent not only intercepts the user 
interface and application calls, the Capture Agent 



records timing information regarding when the caDs 
were sent This allows the Capture Agent to generate a 
script 264 to emulate the user session induding the 
speed in which the user input information and the speed 

5 in which the dient computer responded locally 

The Capture Agent may intercept user interface 
and application calls any number of ways known in the 
art For example, the Windows API calls are intercepted 
by a Windows API call "hoot^ which is provided fbr this 

10 purpose. The SQL API calls may he intercepted by 
renaming the "rear calls so that the SQL API calls go to 
the Capture Agent. After the Capture Agent intercepts 
the SQL calls, the "rear SQL call is then sent so that the 
user session may continue. 

15 The script emulates a user session by being able to 
reproduce the user and application delays. This is more 
comprehensive approach than merely monitoring the 
network traffic. Pr^erakriy, the script is written in a 
source language meaning a programming lar^uage 

20 whk^h includes huntan-readable program statements. 
This allows the scripts to be wore easily edited to vary 
the captured user sessk)n and add contrd constructs. In 
a prefenred embodiment the script indudes program 
statements in the C programming languaga 

25 Fig. 6 shows a Ngh level flowchart of load testing. 
Many of the steps shown are optional and many of the 
steps are more thoroughly discussed in the si^dfica- 
tion that follows. However, this high level f towchart is 
provkf ed to give an overall view before discussing more 

30 detailed aspects of the present invention. 

A user session is begun on a cGent so that the user 
interface and af^lication calls may be captured at step 
302. The calls are captured by the capture agent which 
records timing information regaiding the captured caOs 

95 atstep304. During or after the user session, a saipt is 
generated at step 306 that is capable of directing emu- 
lation of the user session. The Capture Agent generates 
the script wtvch preferably indudes source language 
statements and the timing information of the capture 

40 user interface and application calls. 

At step 308. the script may be edited. Although this 
step is optional, it may be useful to edit the script in 
order to enhance the scr^ (6.g.. add loops) or modify 
the data. For exanple, if a scr^t of a typical user ses- 

4s sion that adds a database record is captured. It may be 
benefidal to edit the script so that the script adds differ- 
ent database records when multiple copies of the saipt 
are executing. Thus, if a user session adds a database 
record for an employee named John Smith, the script 

50 may be edited to add a record for an emptoyee from a 
data file or a random name. In this manner, when the 
script is utilized to emulate, for exaiviple. a hundred 
users, all one hundred users will not attempt to add the 
same datat^ase record. Not only would this run the risk 

55 of causing an error, it would not adequatdy emulate 
realistic multiple user sessions. 

The saipt or saipts are compiled at step 310. In a 
preferred embodin^ the saipts include source lan- 
guage statements and may be compiled into executable 
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programs that emulate user sessions. Load testing may gram to capture user interface and application caQs to 

be perfbrnned with a single scr|pl or wHh multiple scripts. generate a swipt. The Capture Agent generates a script 

Typically, however, multiple scripis are utilized to simu- on the client computer according to the user session at 

late tens or hundreds of users. step 404. The script may include user interface calls. 

At step 31 2. load testing of an appllcatton or system s application calls, and timing infonnation of the calls. The 

under test is performed utilizing the conpiled scripts. Capture Agent is stopped at step 406. The Capture 

The compiled scripts are run by a script driver which is Agent is an important aspect of the present invention 

a computer connected to the server over a network sIm- and its operation wOl be more fully described in refer- 

ilar to or the same as the network that will be utilized to ence to Figs. 9-12. 

connect the clients to the server In actual operation. io The saipt may be edited at step 408. As mentioned 
Fig. 7 shows load testing of a system under test A previously, the script may be edited to insert program 
script driver 352 is connected to a server 354by a net- control statenfients like loops orio alter data so that the 
work. In one embodiment, the saqpt driver is a relatively data more accurately represents user sessions. Adcfi- 
fast computer, workstation or mainframe running the tK>nally. the saipt may be edited for any number of rea- 
UNIX operating system. For example, the saipt driver is sons. Because the script contains source language 
may be a workstation from Sun Microsystems. Inc. statements in pretended embodiments, the saipt is not 
which when mnning UNIX provides multitasking capa- only human-readable but provides the power and capa- 
bilities which are utifized to emulate multiple user ses- biities of the source language (e g., the 0 programming 
sions. language). Additionally, captured application calls for 
The script driver may store multiple compiled so differem applicatwns from different vendors may be 
scripts 356. The scripts are run on the script driver as translated into the same source language, 
multiple processes so that transactions 358 are sent to After the script is edited, the script is transferred to 
the server. Because of the liming infomnation in the the script driver at step 410. The saipt is typk»lly sent 
scripts, the script driver may faithfully emulate the over the network to the script driver but mgy be input 
actions of multple users. Therefore, the server is oper- 25 into the scrip* driver utilizing other means like ftoppy 
ating as if multiple users are utilizing the database appli- disks. 

cation from dient computers on the nelworic. The sender At step 412. the script is compiled on the saipt 

responds by sending results 360 to the saipt driver In driver. Before the script is compiled, it may be njn 

response to the transactions. through a script preprocessor that adds source lan- 

Although Rg. 7 shows a minimal syst^ where only so guage statements or other infonnation to the script to 

a script driver and the system under test are used for make it more suitable for compiling. For example, In a 

k>ad testing, load testing may be performed with any preferred embocfiment where the source language is 

number of client computers on the networfc Isetween' the C progiamming language, the saipts do not contain 

thesalptdriverandthesen^ersothatthedienlsdlsplay compiler preprocessor statenrtents like "#include" or 

user sessions as they are njn. This aspect of the 35 keywords like the curiy braces "{"and T- The script pre- 

present embedment is called "display mode" and wBI processor acids to or othenwse modifies the script so 

be discussed In more detail in reference to Figs. 10. 13 that it is ready to be compSed. 

and 1 4. The script or scripts are utilized to form an executa- 

Fig. 8 shows a flowchart of the process of aeating ble load testing program at step 414. If a single serin is 

an executable foad testing program. Tbe executable 40 to be utilized to emulate a single user session, the saipt 

load testing program is the program that operates on the Itself represents an executable foad testing program for 

script driver to emulate one a typk»lly many users. The the user. However, typically the toad testing program 

process steps are performed on different computers . simiiates multiple users running different user ses- 

including the dients and the script drivers, although dons. A Mix program alfows more than one script to be 

these steps may be performed sequentially on a net- 4S combined to simulate multiple users. The definition of 

work Induding the script driver, dients and sen^r. Many variables for the foad testing program like the number of 

of the steps may be performed on different nelworits. at users, ttte scripts to be utilized for each user and any 

different locations, and at different times. For example. ddays between the execution of scripts may be sped- 

the generation of saipts does not require the script fied. In a prefenred embodiment, the Mix program may 

driver and may be perfonmed on one or numerous dient so be run in a batch mode or interactively, 

computers. As another example, the scripis are As an example, a very simple table may be aeated 

described as being edHed on the dient but they may be in a fde called -4user,tab" to indude the fdtawing: 
edKed. if at all. on any computer that provides saipt 

editing capab9ities (typically ASCII editing). Therefae. userl. scripti 

the actual process of executable load testing program S5 user2. scripti 

will vary according to many factors whfoh will be readily user3. scrqatl 

apparent to those of skin in the art. user4, scr^na 

At step 402. tiie Capture Agent is initialized on a di- 
ent conputer. The Capture Agent is directed by a pro- This table spedfies that there are four users and that 
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three of the users will be ennulated by scripti aiKl one 
user will be emulated by 8cript2 (where "scripti" and 
"8aript2" are the compiled script names). Although the 
compiled scripts may be started interactively either indi- 
viduafly or utitizing a table file like 4user.tab, it is often- 5 
times easier to utilize a batch fie. 

Multiple batch files may be created that utilize the 
4user.tab table file. For example, a batch file may be 
created that starts all the users specified by the scripts 
as follows: to 

use 4user.tab 
start all 
watt 

qutt ' 15 

The first statement directs ttie Mix program to utiize the 
4user.tab table The next three statements direct the 
Mix program to slartcUl the scripts in the table and wait 
for them to finish before quitting. so 

Another batch file may be created that utilizes the 
4us6r.tab tedMe file wtiich starts three of the users spec- 
ified by the scripts five seconds apart thirty seconds into 
the mix session. The batch fie would be as fbllows: 

25 

use 4user.tab 
at 30 start userl 
at 35 start user2 
at40startuser4 

wait 30 
quit 

As before, the first statement directs the Mix program to 
utilize the 4user.tab table. The next three statements 
specify the number of seconds from the t>eginning of the ss 
mix session when userl, userS and user4 should be 
started. Thus, thirty seconds after the mix session 
begins, userl , user2 and user4 will be started five sec- 
onds apart The wait and quit statements direct the Mix 
program to wait for the started users to finish before 40 
quitting. 

The preceding were just a couple of sinple exam- 
ples of the power that the Mix program provides. When 
used interactively, the Mix program allows simple load 
tests to be quicMy initiated. The batch fOe capability 45 
allows more complicated load tests to be aeated and 
saved for future use. 

Figl 9 shms a high level flowchart of the Capture 
Agent Capture is initiated by a user at step 402 of Fig. 
8. For ease of refererKe. this same step is shown as so 
step 452. After the user starts capture, the Capture 
Agent is launched at step 453. The Capture Agent is 
directed by a computer program called the Capture pro- 
gram. 

At step 454. the Capture Agent may check the ss 
license on the script driver. This step is optional but may 
be utilized to ensure that the software on the script 
driver is licensed software. In one embodiment, the 
script driver software is Bcensed only for particular com- 



puters. The Capture Agent then verifies that the license 
on the script driver matches the script driver computer. 
If the license is not verified, the Capture Agent will not 
proceed. 

The Capture Agent installs hooks to capture user 
interadions and database functions at step 456. Thus, 
the Capture Agent installs or sets up whatever hooks or 
interc^on mechanisms are needed to capture user 
interface and application calls. The "hook" API call may 
be utilized to capture Windows API calls and renaming 
SQL API calls may be utilized to capture SQL API calls. 
Other mechanisms may be utilized depending on the 
nature of the computer architecture, operating syston, 
network operating system, and application to be tested. 

At step 458, the Capture Agent monitors user inter- 
face and application calls to generate a script. The Cap- 
ture Agent captures or intercepts the calls and timing 
information of when the calls were sent. In this manner, 
the generated script is able to reproduce the user ses- 
sion including the tinning of the calls. The process of 
generating the script from the captured user interface 
and application calls will be descrbed in more detail in 
reference to Fig. 10. 

Capture is stopped by a user at st^ 406 of Rg. 8. 
For ease of referenca this same step is shown as step 
460. After the user stops capture, the Capture Agent 
uninstalls the hooks to capture user interactions and 
database functions at step 453. The hooks are unin- 
staled so that the client will be more m the condition 
before the Capture Agent was launched. The Capture 
Agent exits or ceases to run at st^ 464. 

Rg. 10 shows a process the Capture Agent per- 
fonns to generate a saipt from the capture calls. At step 
502. the user interface and applicatton calls are cap- 
tured. Timing irtfbrmatk)n regatding when the calls were 
captured is recorded at 504. The timing Infonnation may 
include or be used to cakxjiate think time for user inter- 
face calls and application processing time for applica- 
tion calls. 

The term "think time* is used to refer to the time 
starling when the computer is able to accept input from 
a user and ending wtien the user enters (via the enter 
key or utilizing mouse txjttons) commands or data (e.g.. 
an e^ent). For example, if it takes 75 seconds for a user 
to input information or otherwise produce a Windows 
API call, then a think time of 75 seconds may be 
recorded. In a preferred emtxxJiment, the present inven- 
tion provkJes many options relating to thlrk time. For 
example, a uniform think time may be specified at the 
beginning of the saipt so ttiat there are less time pres- 
sures during a user script that generates a script. Addi- 
tionally. the Capture Agent may be instructed to only 
add think times H the think time is longer than a speci- 
fied amount of time. 

The term "application processing time* is used to 
refer to the time starting when a user interface call is 
received from the user and ending when the client appli- 
cation responds locally, for example by redrawing a win- 
dow on the display. The application processing time 
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does not refer to the time that the server takes to proc- 
ess an SQL call. For example, if It takes 0.4 seconds tor 
the cGent to bring up a window used for fetching data, 
then an application processing time of 0.4 second may 
be recorded. 

The think and application processing times are uti- 
lized to emulate a user session in "non-display nrKXie". 
In non-display mode, the saipt driver communicates 
directly with the server as shown in Fig. 7. Because a 
client computer is not performing the client application 
functions, the user input to the client applicalk)n is not 
utilized. Instead, the think and applicalion processing 
times are utilized. 

In display mode, at least one client computer 
accepts the user input to the client application and 
reproduces the screen displays,. This arrangement is 
shown in Fig. 14. Thus, in display mode, the script driver 
communk»tes with the server through a client computer 
which may be very useful during load testing of a sys- 
tem. Display mode or non^lisplay refers to a single 
emulated user session or saipt Consequently, some 
scripts may fc>e run In display nnode while other scripts 
are run in non-display mode. Of course, display mode 
requires that a client be availabie so the actual number 
may be linvted by the hardware available. 

In order to illustrate the effect of display mode and 
non-display mode on the saipt the foHowing script seg- 
ment will be analyzed: 

Think(2.026): 

LeftButlonPre68(459. 616); 

AppWart(0.22): 

WindowRcv("OwPtR^: 



The Thirik statement indicates the user took 2.026 sec- 
onds to dick on the left mouse button as indicated by 
the following LeftButtonPress statement. The I^But- 
tonPress statement indicates that the user pressed the 
left mouse button at those coordinates. The Appwait 
statement Indcates the client computer took 0.22 sec- 
onds to redraw a window as indicated the following 
WindowRcv statement The WIndowRcv statement indi- 
cates that certain windowing events were taken in 
response to the LeftButtonPress. The parameters of the 
WindowRcv function are standard Windows two-letter 
mnemorvcs (e.g., *Cw" means aeale window and "Pt* 
means paint). 

\A/hen the above saipt is run in display mode, the 
LeftButtonPress conmand will be sent to a client conv 
puter after the indicated think time and the WIndowRcv 
command instructs the saipt driver to wait until the indi- 
cated windowing events occur. As the salpt Is achally 
driving a client computer and waiting for the client appli- 
cation to perfomi. the AppWait hinctton is ignored. 

When the above script is run in non-cfisplay mode, 
only the Think and AppWait statements are executed. 
The saipt driver waits the appropriate times to emulate 



the user session but the user interface calls and any 
responses to them are not utilized. All the scripts in Fig. 
7 wouki be run in non-display nxxJe as there is no dient 
to display a saipt in progress. In Ftg. 14. scripts may be 

5 njn in display and non-display mode as there is at least 
one client to display a saipt in progress. 

Additionally, the Capture Agent may t>e instructed 
to only capture the application calls (e.g., SQL API 
calls). This may be done to conserve space in the script 

10 files. Thus, the above script would not include the Left- 
ButtonPress statement Howa/er. the Think statement 
would still tie present so that the script delays time 
associated with user input time. The script may then 
only be run in non-display nx)de as the user Inteifaoe 

15 calls are not present in the saipt 

Although a user rnay physicaOy operate a dient 
oomputer while generating a saipt for a user sessfon, 
the present inventfon may also be utilized with a GUI 
tester. A GUI tester is a program that takes over the 

20 inputs to an appfication in order to test it The Capture 
Agent may be utilized with a GUI tester so that the GUI 
tester operates the dient application and the Capture 
Agent generates a script. Thus, the present invention 
may t>e used in cor^unction with GUI testers and does 

25 not require that a user operate the client to generate a 
script for a user session. Accordingly, the terms "user 
session", "user interface call" and the like, do not Bnply 
that a living user must be operating the dient. 

Still referring to Fig. 10. pointers In the applteation 

30 calls to dient data are dereferenced at step 506. Many 
applicatfon calls indude references or pointers to data 
that is stored tocally on the dient The Capture Agent 
dereferences these pointers (accesses the data refer- 
enced) and places the data in the saipt For example. 

35 the foltowing is a script segment Inducfing SOL caHs: 

Open(LOGI.CURI) 

Parse(CUR1. "INSERT INTO CUSTOMERS (ID. 

40 FIRST^NAME. tAST.NAME, ADDRESS.UNE J , 
ADDRESS_LINE_2. ADDRESS.UNEJ. 
PHONE_NUMBER. FAX^NUMBER. 
COMM_PAID.YTD. ACCOUNT_BALANCE. COM- 
MENTS) VALUES (lid. lhame. :lname. :«ldr1. ^ 

45 3ddr2. addrS, phno. :faxno. :cytd. :bal. rconrvn)'); 
CSset(CUR1. MAXARRSIZE. 1); 
Bind(CUR1. "kf. STRING. 40): 
Bind(CUR1. "ihame-. STRING. 21): 
Bind(CUR1. "ilname". STRING. 21): 

so Bind(CUR1. "laddrl". STRING, 21): 
Bind(CUR1. ":addr2-. STRING. 21): 
Bind(CUR1. 'raddrS". STRING. 21): 
Bind(CUR1/:phno". STRING. 16): 
Blnd(CUR1. "tocno". STRING. 16): 

55 Bind(CUR1. •:cytd\ STRINa 40); 
Bind(CUR1. ^bal". STRING. 40): 
Bind(CUR1. "rcomm". STRING. 241): 
Dala{CUR1. "555|John|Smith|123 Elm St|Any- 
town. MD 12345|USA|555-555|555- 
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5555|9000|5000|Noxx>mrnems*); 

Exec(CUR1); 

aose(CUR1); 



The above script include source language statements in 
the C programing language The statements acid a new 
data record John Smith into CUSTOMERS. The state- 
ments correspond very dosely to actual SOL API calls. 
However, in actual SQL API calls, the Bind statements 
may have pointers to the data for John Smith on the di- 
em computo^ (e.g., the Bind statemerits may have point- 
ers as parameters). 

The Capture Agent identifies the pointers, derefer- 
ences the pdnters to access the data, and automati- 
cally generates the Data statement which includes the 
accessed data. In this way. the script driver is able to 
replicate the. user session without requiring access to 
the dient^s memory, whidi is typically unavailable dur- 
ing load testing. Another advantage is that by derefer- 
endng the pointers and pladng the accessed data in 
the ea^. the scrpt may be more easfly edted so that 
each copy of the script will add drflerent customers to 
the database when run. This may be done by fHiing in 
the data statement from a file or using some kind of ran- 
dom data. 

At step 508. the script is generated. The script may 
be generated during and after a user session, in other 
words, the script may be generated alihe end ol a cap- 
tured user session, during the captured user session, or 
tx>th. The script is typically saved onto the hard drive of 
the dient computer. 

Fig. 11 is a typical captured database session. 
Once the dient appication of the database application 
is running on the cfient, a Think timer is started at step 
552. The user inputs data at step 553 until a terminating 
character (e.g^, the enter key) or action (ag.. cfiddng a 
mouse button) has been taken. 

After tiie user is finished infUJtting data, the Think 
timer is stopped aft 554. The user input in the assodated 
user interface caH is recorded at step 555. If the user 
input indicates the user wants to quit the application at 
step 556, the application terminates. Otherwise, an 
Application Processing timer is started at step 558. 

At step 559, the user input in the form of the user 
interface call is sent to the cfient application. Once the 
dient appGcation receives the user interface call, the 
application peribrms whatever processing is required at 
step 560. For example, the user interface call noay spec- 
ify to perform a database function as shown in step 562 
or to display results in step 564. The database function 
typically requires the dient application to send a request 
to the server via tiie network The processing of a data- 
base function will be desaibed in more detail in refer- 
ence to Rg. 12. 

The dient application continues to process the user 
interface call until all the processing has l>een done. 
The processing has been completed at step 566. 

Rg. 12 shows processing of a database function. At 



step 602, the dient application calls a datable function 
(i.e.. an application call). In a prefened emt>odimem, the 
database function is in the form of an SQL API call and 
the dat^se function calls a substitute database func- 
5 tion SO that the Capture Agent can capture or intercept 
the call. The substitute database function is called at 
Step 604. 

The Application Processing timer is stopped at step 
606. Information being sent to the database server in 

10 ttie database function is saved at st^ 608. The informa- 
tion indudes the database call and any data that is ref- 
erenced through pointers in the call. At step 610, the 
real daftat>ase function is called. 

At step 612, a determination is made as to whether 

75 the database function resulted in an error from the 
sen/er. For example, the error may be cause by an 
incorrect a ill-fonmed database functfon call. Howo/er. 
when capturing a user sessran, a user may generate an 
error. In order to reproduce the user session, the script 

so should also do the same actions to generate the error. 
These errors though are expected so the script sets an 
enror condition (e.g.. a flag) to COMTINUE at step 618. 
When the script later encounters this error during exe- 
cution, the script will check the error condition to see if it 
.25 is set to COfTTINUE. If the error condition is set to CON- 
TINUE, the script knows that the error was expected 
and continues. 

K the database function did not result in an error 
from flie senrer, the error condition is set to EXIT at step 

30 620 which instructs the script to exit if it encounters an 
. enor because it is unexpected. The error condition does 
not have to be set for each database function but may 
only be set when the enror condition needs to be 
changed. Typically, most database functions will not 

35 generate an ena^so the error condition will usually be 
set to EXIT. The script will then exit when an unexpeded 
en'or occurs during scrpt execution. 

At step 622. data coming back from flie database 
server (e.g.. results) are saved. Script statements are 

40 then generated at st^ 624. Once the script statements 
are generated, the Applicatkxi Processing timer is 
started at step 626 and control Is returned to the dient 
application at step 628. 

Rg. 13 shows a ftowchart of perfornrting load test- 

45 ing. The f kiwchart assumes that there are already com- 
• piled scripts resident on the script server. At step 652. a 
det&mination is made whether the load test should be 
performed in display mode. In order for the load test to 
be performed in diqalay mode, there should be at least 

50 one dient computer on the networicas shown in Rg. 14. 
For each dient that will display the script in progress, a 
Display program is started on the dient at step 654. 

The Display program is designed to accept input 
from the script driver to simulate a live user. The irqxit is 

65 generated tiy the saipt and was typk^lly captured from 
a previous user session. 

At step 656. the Mix program is run. The Mix pro- 
gram allows, either interactively or in t>atch files, more 
ttuin one script to be executed on the script driver. The 
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Mix program is opttonat but is typically run to simulate 
multiple users interacting with the server. A flag or 
parameter is set for each script that wit) be executed in 
display mode so that the script driver sends the appro- 
priate inputs to the client 

Each script that executes on the script driver cre- 
ates a log file at step 660. A report may be generated 
from the log files which includes information such as 
server response time and system throughput. In a pre- 
ferred enft)odiment, the log files are ASCII files on the 
server so that third party statistical and graphics pro- 
grams may be utilized to aeate custom reports. 

At step 664. a determination may be made whether 
to rerun another load test. For simplicity, the flowchart 
shows a return to step 656 to run the Mix program and 
possitsly select a different set of users, number off 
users, user start times, and the Eke. However, this 
should not inrply that a retum to many of the other steps 
is not possitila For example, a different cHent may be 
selected to run the display program at step 654 or a new 
user session may be captured on the client at step 402 
of Fig. 8. 

Rg. 14 shews load testing of a system urxjer test 
suitable for display a non-display mode. A script driver 
702. clients 704 and a server 706 are connected by a 
network. The script driver may store multiple compiled 
scripts 708. Scripts that are run in display mode send 
user interface calls 710 to one of the clients 704. The cli- 
ents then disptey the saeen displays that occur during 
the emulated user session and also send transactions 
712 to the server. The server sends results 714 to the 
clients in response transactions 712. The clients 704 
fonward results 714 to the script driver. 

Scripts tliat are run in non-display nnode send trans- 
actions 718 directly to the sen/er. the server sends 
results 720 to the saipt driver in response to transac- 
tions 720. Transactions 71 2 and 71 8 do not differ except 
lor tfie source of origin of the transactions t>eing the 
script driver or dients. respectively. 

Fig. 15 shows a flowchart of a process of emulating 
a user session from a saipL The scripts execute on the 
8crq;>t driver with each scrpt representing a user ses- 
sion that was typically captured by the Capture Agent 
In a preferred embodiment, the scripts are compiled and 
include source language statements so the execution of 
a script is the execution of a computer progrant The 
scripts may contain different control statements includ- 
ing and branches so there is no "one" process 
flow of a script. The script process in Fig. 15 is pre- 
sented to illustrate a simple saipt. 

At step 752. the script starts and begins executing 
the compiled source language statements. If there are 
more functions or statements at step 754. the functions 
are executed. 

A datat>ase function is specified at step 756. The 
datat)ase function is translated into the appropriate 
applicafion call (e.g.. SQL API call) which is sent to the 
server via a network transport layer. 

User processing is specified at step 760. The user 



processing may include Think delays, Pointer (e.g.. 
mouse) delays or Typerate (speed of typing) delays. 
The script may have any of these delays globally 
defined at the beginning of the script. If the script is exe- 
5 cuted in display mode, the specified user interface call 
or calls will also be executed or sent at step 762 after 
the delays. 

Application processing is specified at step 764. An 
application processing delay (e.g.. AppWait) will be per- 

10 formed at step 766 if the script is executed in non-dis- 
play mode. If the script is executed in display mode, the 
script will wait for the diem to return from generating the 
window displays. 

Once all the functions or statments in the script 

15 have been executed, the script creates a log file at step 
768. The log file may be aeated and written to as the 
script executes but the step is sixwn at the end for sim- 
plicity. After the log fi e has been created, the scrqpt exits 
at step 770. 

20 The a!)ove description is illustrative and not restric- 
tive. Many variations of the invention will become appar- 
ent to those of skill in the art tpon review of ttas 
disclosure. Merely by wa^ of exanple the inventton is 
illustrated wHh regard to database applicattons. but the 

25 invention s not so imited. The scope of the inventkxi 
should, therefore, be determined not with reference to 
the akX3ve desalptkm. but instead should be deter- 
mined with reference to the appended claims along with 
their full scope of equivalents. 

30 

aalms 

1. hi a computer system ftaving a server computer 
and a client computer, a method of producing 

35 scripts for toad testing a software application, com- 
prising the steps of: 

capturing application calls on the client compu- 
ter, the application calls including application 
40 calls generated in response to user interac- 

tions; 

recording timing information of the captured 
application calls; and 

generating a script from the captured applica- 
45 tion calls that generates application calls 

accoiding to the timing information of the cap- 
tured application calls. 

2. The method of daim 1, further comprising the steps 
so of: 

capturing user interface calls on the client com- 
puter; 

recording timing infornf^ation of the captured 
55 user interface calls; and 

generating the script from the captured user 
interface calls that generates user interface 
calls according to the timing information of the 
captured user interface calls; and/or further 
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comprising the step of executing a plurality of 
saipts to emulate a plurality of users: 
and/or further conprising the step of creating a 
report of response times of the server oonpu- 
ter. 

3. The method of dairn 1 or 2. further comprising the 
step of setting aflag in the script if an error is gen- 
erated during script generation; 

and optionally further comprising the step of 
ignoring an error generated during script execution 
if the flag indicates the enror was also generated 
during script generation. 

4. A computer progrsun product that produces scripts 
for load testing a software application, oorrprising: 

computer code that captures application calls 
on a client computer, the application calls 
including application calls generated in 
response to user Interactions; 
computer code that records timing information 
of the captured application calls; 
computer code that generates a script from the 
captured application calls that generates appli- 
cation calls according to the timing information 
of tfie captured application calls; and 
a computer readal)l6 medium that stores the 
computer codes. 

5. In a computer system having a server computer 
and a client computer, a method of producing 
scripts for load testing a software application, com- 
prising the steps of : 

capturing application caDs.on the client compu- 
ter, the captured application calls including ref- 
erences to data stored locally on the client 
computer; 

identifying in the captured application calls ref- 
erences to data stored locally on the client 
computer; 

accessing tiie data ttvough the references in 
the captured application calls; and 
generating a script from the captured applica- 
tion calls that generates appficafion calls that 
include the accessed data in place of the refer- 
ences in the captured appGcation calls, the 
saipt Inducting the accessed data. 

6. The method of daim 5. wherein ttie script includes 
source language statements; 

aiTKl optionally further comprising tiie step of 
compiling tiie script into an executable load test 
program. 

7. The mettiod of daim 5 or 6, further comprising the 
step of load testing a system utilizing a plurality of 
scr^ris. 



8. A computer program product ttiat produces scripts 
for load testing a software application, comprising: 

computer code that captures application caOs 
5 on the client computer, the captured application 

calls including references to data stored locally 
on the client computer; 

computer code ttiat identifies in the captured 
application calls references to data stored 

10 locally on ttie dient computer; 

computer code ttiat accesses the data through 
the references in ttie captured application calls; 
computer code ttiat generates a script from ttie 
captured application calls that generates eppti- 

IS cation calls that include ttie accessed data in 

place of the r^erences in ttie captured applica- 
tion C€Uls, the script including the accessed 
data; and 

a computer readat)le medium ttiat stores ttie 
so computer codes. 

9. In a computer system having a server computer 
and a dient computer, a mettiod of producing 
scripts for load testing a software application, com- 

2S prising the Steps of: 

capturing appBcation calls on ttie client compu- 
ter, the captured applicalion calls specifying 
information to be sent to ttie server computer; 
30 translating the captured appltcation calls into 

source language statements; and 
generating a saipt induding ttie source lan- 
guage statements that generates application 
calls. 

35 

10. The method of claim 9, further comprising the step 
of translating captifl'ed applicalion calls for different 
applications imo a same source language. 

40 11. The mettnod of claon 9 or 10, further comprising ttie 
step of capturing user interiace calls on the dient 
computer; 

and optionally furttier comprising tiie step of 
translating the captured user interface calls into 
45 source language statements, wherein, optionally, 
ttie script generates user interface calls. 

12. A computer program product that produces scripts 
for load testing a software application, comprising: 

so 

computer code that captures application caQs 
on the client computer, ttie captured application 
calls specifying information to be sent to the 
server computer: 
55 computer code ttiat translates the captured 

application calls into source language state- 
rrients; 

computer code ttiat generates a script includ- 
ing the source language statements that gener- 
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ates application calls; and 

a computer readable medium that stores the 

conputer codes. 

13. tn a networked computer system having a first com- s 
puter (saipt driver) and a second computer (system 
under test), a method of for load testing a software 
application, comprising the steps of: 

the first computer executing scripts that emu- 10 
late a plurality of users, the scripts including 
delays representative of actual user sessions, 
the first computer sending requests to the sec- 
ond corrputer over a network; 
the second computer responding to the 15 
requests over the networt^; and 
measuring response times of the second com- 
puter. 

14. The method of claim 13, further comprising the step 20 
of a third computer on the network displaying a 
script directed user session in progress. 
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